Why liquid-cooled infrastructure has become mainstream – and how it can
Once considered niche, liquid-cooled infrastructure has become a much more common solution to maintain power-hungry components such as graphics processing units (GPUs) within data centers. Many cite the AI ‘gold rush’ as the driver of liquid cooling systems, with data center operators clamoring for ways to cool expanding data centers. But it’s not the only reason this kind of cooling solution is growing in appeal.
“AI’s the one that gets all the publicity, but there’s a whole bunch of other areas driving uptake,” says Tony Lock, director of engagement and distinguished analyst at Freeform Dynamics.
“Take real-time analytics for example – organizations are increasingly adopting this approach by integrating multiple systems into a single, more powerful framework that processes data faster and more efficiently.”
Liquid cooling is able to ‘handle the heat’ produced by the power-hungry technologies driving infrastructure innovation. This ability to manage generative AI workloads, edge computing, high-density applications and growing storage demands better than many traditional cooling systems, is why Gartner identified it as a top trend impacting infrastructure and operations this year.
Jeffrey Hewitt, VP analyst at Gartner, says that liquid cooling has evolved to move from cooling the broader data center environment to getting closer and even within the infrastructure via direct to chip (D2C) cooling. While its scope is limited today, it will become more prevalent as next generations of GPUs and CPUs increase power consumption and heat production.
Fellow analyst firm Omdia agrees that liquid cooling is becoming the standard for cutting-edge AI chips and servers deployed at scale, citing AI training, high performance computing (HPC) and cryptocurrency mining as the main applications right now.
Wang Shen, principal analyst, Data Center Power & Cooling Systems at Omdia, tells ITPro that while traditional air cooling is being challenged, in the short term we’re likely to see chillers, air handling units (AHUs) and perimeter continue to function alongside liquid cooling facilities.
“These traditional air-cooling systems can support liquid cooling solutions well by providing chiller water, cooling the environment – especially when liquid-to-air coolant distribution units (CDUs) are installed – or cooling down other components that are not covered with cold plates – such as PSI memory and hard drives.”
The advantages of liquid cooling
“Liquid cooling offers a number of advantages. Its cooling capacity is much larger than that of air cooling within the same space, for example, plus it’s quieter, and more stable and controllable,” says Shen.
In addition, it requires less space in servers and racks, enabling a more flexible mechanical design, which leads to higher deployment density.
Energy efficiency is one of the big benefits, particularly in terms of total cost of ownership (TCO), but there is a high outlay to consider when introducing liquid cooling, as Shen explains.
“Liquid cooling has higher upfront costs but reduces energy consumption and operational expenses in the long term while extending hardware lifespan. Environmentally it is more energy efficient, though some systems may consume water, requiring attention to sustainability.”
“There’s a lot of reports coming out of the US of tech companies using groundwater and that’s only going to increase as more of these sites pop up,” says Omer Wilson, CMO at Smart Campus. “Therefore the industry is having to look at ways to improve the sustainability of these kinds of solutions.”
Cooling with a conscience
Companies are looking at ways to make liquid cooling more sustainable in terms of water use. Take Microsoft’s closed-loop liquid cooling system for example. Announced in December 2024, the tech giant’s new data center design continually circulates water between servers and chillers to dissipate heat without requiring a fresh water supply. It expects this will save 125 million liters of water per data center, per year.
This follows the Microsoft’s cancelled Project Natick, in which the firm submerged prototype data centers in the North Sea to make use of lower underwater temperatures for reduced cooling costs. Though it had the added benefit of bringing data centers closer to coastal communities and thereby reducing latency, the project was ultimately shelved due to the extreme difficulty of servicing underwater equipment.
Some have instead sought to harness seawater for use in more recognisable cooling technology. One of the first organizations to do this at scale is Smart Campus, which has repurposed a former power plant’s existing ocean water intake and outtake infrastructure to provide a large-scale ocean water cooling system similar to that of a closed loop solution.
However, recycling water from the Atlantic comes with many regulatory challenges.
“You have to – understandably – go through a lot of regulatory loops to ensure that the state of the water we put back into the ocean in terms of temperature, quality etc is at a certain level,” says Wilson. “But in this scenario, you’re not pulling fresh water, contaminating it and then having to get rid of it as waste. You’re basically recycling seawater, so it’s sustainable in the truest sense.”
Lock also points to data centers finding positive ways of re-using waste heat.
“Heat reclamation technologies have evolved enormously in the last 20 years. With liquid cooling you’ve essentially got smaller volumes of heat that are easier to transfer to re-use somewhere else. I’ve seen it used to heat a building, a local swimming pool. In Scandinavia I’ve even seen it used to heat homes and buildings within a local district heating system.”
Is liquid cooling right for your infrastructure?
While this all sounds positive, high initial costs, skills gaps, system complexity, leakage risks, retrofitting challenges and regulatory restrictions will likely put many IT leaders off liquid cooling systems. Furthermore, as Lock highlights, air-cooling systems are still great for many day-to-day setups.
Unsurprisingly today the biggest users of liquid cooling are the hyperscalers, but how do you go about deciding whether liquid cooling’s the right solution for your organization’s infrastructure needs?
Lock advises you work out if the end benefits outweigh the high start-up costs, noting that if you’re running enough intense workloads then the power savings liquid cooling provides can cover the upfront costs.
“If you’re a standard enterprise you might want to focus on introducing it for certain areas, such as AI or high-speed computing for real-time financial transaction processing; the things you want to run fast and hot,” he adds. “You also need to consider how much work has to go into setting up the plumbing to enable such a system.
“You also want to consider how your company’s financed,” he continues. “If your investors expect a return in three months then it’s probably not the option for you. But if you’re looking at five or ten years, that’s different. The answer depends on many factors.”
We’re yet to reach the epitome of what liquid cooling can offer. Analysts note that innovations around eco-friendly coolants, two-phase cooling, pump-free liquid cooling, AI-optimized cooling systems and modular designs have the potential to further reduce the environmental impact of liquid cooling, making it an even more appealing option.
For those considering dipping their toes in the water either now or later, Wilson ends with this advice:
“Detailed research is your friend. This area is changing rapidly with lots of innovation coming out of a lot of different countries. Keep an eye on the companies developing innovative new ideas. Look at their case studies and examples of what they’re doing to see how they might fit with what you need.”
Source link
